<html>
<head><meta charset="utf-8"><title>Binary search format · rustdoc · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/266220-rustdoc/index.html">rustdoc</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/266220-rustdoc/topic/Binary.20search.20format.html">Binary search format</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="223682276"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/266220-rustdoc/topic/Binary%20search%20format/near/223682276" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/266220-rustdoc/topic/Binary.20search.20format.html#223682276">(Jan 22 2021 at 18:46)</a>:</h4>
<p>So I was looking at perf for <a href="https://github.com/microsoft/windows-rs/issues/420#issuecomment-765549226">https://github.com/microsoft/windows-rs/issues/420#issuecomment-765549226</a> and one of the issues is the search index is almost 60 MB (!!). There's an issue open for fix it (for almost 5 years now): <a href="https://github.com/rust-lang/rust/issues/31387">https://github.com/rust-lang/rust/issues/31387</a>. Some of the suggestions were:</p>
<ul>
<li>Use smaller names in the JSON keys</li>
<li>Use a binary format</li>
<li>Somehow load the information lazily instead of eagerly</li>
</ul>
<p>I think "use a binary format" is the simplest, and should also have a big effect on the file size and load time. <span class="user-mention" data-user-id="210316">@GuillaumeGomez</span> what do you think? Do you have suggestions for a format?</p>



<a name="223683569"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/266220-rustdoc/topic/Binary%20search%20format/near/223683569" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> GuillaumeGomez <a href="https://rust-lang.github.io/zulip_archive/stream/266220-rustdoc/topic/Binary.20search.20format.html#223683569">(Jan 22 2021 at 18:58)</a>:</h4>
<p>Your guess is as good as mine. However, please keep in mind that we need to be able to manipulate it easily to add or update a crate in the search index.</p>



<a name="223684404"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/266220-rustdoc/topic/Binary%20search%20format/near/223684404" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Dániel Buga <a href="https://rust-lang.github.io/zulip_archive/stream/266220-rustdoc/topic/Binary.20search.20format.html#223684404">(Jan 22 2021 at 19:05)</a>:</h4>
<p>Would a compressed json (using gz or something) be terrible?</p>



<a name="223684575"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/266220-rustdoc/topic/Binary%20search%20format/near/223684575" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/266220-rustdoc/topic/Binary.20search.20format.html#223684575">(Jan 22 2021 at 19:06)</a>:</h4>
<p>hmm, I'm not sure if that works without a web server</p>



<a name="223684585"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/266220-rustdoc/topic/Binary%20search%20format/near/223684585" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/266220-rustdoc/topic/Binary.20search.20format.html#223684585">(Jan 22 2021 at 19:06)</a>:</h4>
<p>doing all this locally makes it hard :/</p>



<a name="223684701"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/266220-rustdoc/topic/Binary%20search%20format/near/223684701" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Dániel Buga <a href="https://rust-lang.github.io/zulip_archive/stream/266220-rustdoc/topic/Binary.20search.20format.html#223684701">(Jan 22 2021 at 19:07)</a>:</h4>
<p>I'm not sure if it's any more complicated than with a server that serves a .gz, but I'm not sure if js implementations of any inflate algo would be fast enough.</p>



<a name="223684771"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/266220-rustdoc/topic/Binary%20search%20format/near/223684771" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Dániel Buga <a href="https://rust-lang.github.io/zulip_archive/stream/266220-rustdoc/topic/Binary.20search.20format.html#223684771">(Jan 22 2021 at 19:08)</a>:</h4>
<p>anyway, probably a bad idea <span aria-label="smile" class="emoji emoji-1f642" role="img" title="smile">:smile:</span></p>



<a name="223684783"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/266220-rustdoc/topic/Binary%20search%20format/near/223684783" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/266220-rustdoc/topic/Binary.20search.20format.html#223684783">(Jan 22 2021 at 19:08)</a>:</h4>
<p>I would be interested in seeing numbers for gzip</p>



<a name="223684814"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/266220-rustdoc/topic/Binary%20search%20format/near/223684814" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/266220-rustdoc/topic/Binary.20search.20format.html#223684814">(Jan 22 2021 at 19:08)</a>:</h4>
<p>are you interested in writing that up? :)</p>



<a name="223684873"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/266220-rustdoc/topic/Binary%20search%20format/near/223684873" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Dániel Buga <a href="https://rust-lang.github.io/zulip_archive/stream/266220-rustdoc/topic/Binary.20search.20format.html#223684873">(Jan 22 2021 at 19:09)</a>:</h4>
<p>unfortunately I'm more interested in figuring out why the keccak benchmark borrow-checks 3300 times <span aria-label="frown" class="emoji emoji-1f641" role="img" title="frown">:frown:</span></p>



<a name="223685004"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/266220-rustdoc/topic/Binary%20search%20format/near/223685004" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> GuillaumeGomez <a href="https://rust-lang.github.io/zulip_archive/stream/266220-rustdoc/topic/Binary.20search.20format.html#223685004">(Jan 22 2021 at 19:10)</a>:</h4>
<p>To be perfectly honest: I think that if it's not something handled by the web-browser directly, the size will reduce, but the performance will drop too.</p>



<a name="223685041"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/266220-rustdoc/topic/Binary%20search%20format/near/223685041" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> GuillaumeGomez <a href="https://rust-lang.github.io/zulip_archive/stream/266220-rustdoc/topic/Binary.20search.20format.html#223685041">(Jan 22 2021 at 19:11)</a>:</h4>
<p>We can add a compressed JSON that we decompress in the browser, but that will very likely be horribly slow</p>



<a name="223685060"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/266220-rustdoc/topic/Binary%20search%20format/near/223685060" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> GuillaumeGomez <a href="https://rust-lang.github.io/zulip_archive/stream/266220-rustdoc/topic/Binary.20search.20format.html#223685060">(Jan 22 2021 at 19:11)</a>:</h4>
<p>our only viable options are the ones provided by the browsers' API</p>



<a name="223685122"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/266220-rustdoc/topic/Binary%20search%20format/near/223685122" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> GuillaumeGomez <a href="https://rust-lang.github.io/zulip_archive/stream/266220-rustdoc/topic/Binary.20search.20format.html#223685122">(Jan 22 2021 at 19:11)</a>:</h4>
<p>Take a look at my blog post I posted today to give you a little idea of the things I went through already ;)</p>



<a name="223685214"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/266220-rustdoc/topic/Binary%20search%20format/near/223685214" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> GuillaumeGomez <a href="https://rust-lang.github.io/zulip_archive/stream/266220-rustdoc/topic/Binary.20search.20format.html#223685214">(Jan 22 2021 at 19:12)</a>:</h4>
<p>but since I'm not a front-end engineer, it's very possible that I missed a way to do things better, which is why I'd be glad if someone came with someone I never heard of (it'd be really wonderful)</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>